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Abstract. Employing the allegorical imagery from the film 
"The Matrix", we motivate and discuss our 'Cyborg Astro- 
biologist' research program. In this research program, we 
are using a wearable computer and video camcorder in or- 
der to test and train a computer- vision system to be a field- 
geologist and field-astrobiologist. 



1 Matrix Preamble 

We choose to use the allegorical symbolism from the film, 
The Matrix (Wachowski & Wachowski, 1999). (The speaker 
emphasized the bold-faced words, and spoke very slowly 
and with a deep, precise voice, like the Matrix character, 
Morpheus, did.) 

You also have a choice. You may choose between the 
blue pill and the red pill (see Figure 1). 
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Fig. 1. You have a choice between the blue pill on the right and the 
red pill on the left. (This picture and all the following Matrix pic- 
tures are from the original Matrix film (Wachowski & Wachowski, 
1999).) 

If you should choose the blue pill, the next robots that you 
send to Mars to search for life, will have no or little A.I. 
(Artificial Intelligence) in them. All the science that these 



robots will do will be done by astrobiolo gists and geologists 
such as yourselves, here on the Earth, after the data has 
been telemetried by the robots from Mars back to Earth. If 
you should choose the blue pill, all of the intelligence will 
remain here on the Earth (see Figure 2). 

However, if you should choose the red pill, these robots, 
bound for Mars, will have on-board, scientific, astrobiolog- 
ical Artificial Intelligence. This A.I. will allow you and the 
robots, together, to accomplish much more science, espe- 
cially given the several minutes of delay for delivery of com- 
mands from the Earth or for delivery of data from Mars (see 
Figure 2). 




Fig. 2. When we explore Mars, you have a choice between keep- 
ing all the Geologist intelligence in human form here on the blue 
planet, the Earth (on the right), and sending significant Geologist 
intelligence in robotic form to the red planet, Mars (on the left). 

If you should choose the red pill, some of our intelligence 
will be bound for Mars... 



2 Live Demonstration of Cyborg Geologist & Astrobi- 
ologist System 

In the Mars Exploration Workshop in Madrid, we demon- 
strated to you some of the early capabilities of our 'Cyborg' 
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Geologist/Astrobiologist System. We are using this Cyborg 
system as a platform to develop computer- vision algorithms 
for recognizing interesting geological and astrobiological 
features, and for testing these algorithms in the field here 
on the Earth. 




Fig. 3. Battery lifetime is important both for running the ruling- 
machines in the Matrix and for reducing the fatigue of the Human 
Geologist who is carrying the wearable computer (and also its bat- 
teries) for the Cyborg Astrobiologist project. Battery lifetime was 
one of the main reasons for choosing a wearable computer from 
ViA, since this wearable computer contained a high-speed, low- 
power 'Crusoe' CPU. 

One advantage of this Cyborg approach is that it uses hu- 
man locomotion and human-geologist intuition/intelligence 
for taking the computer vision-algorithms to the field for 
teaching and testing, using a 'wearable' computer. We con- 
centrate on developing the 'scientific' aspects for autonomous 
discovery of features in computer imagery, as opposed to 
the more 'engineering' aspects of using computer vision to 
guide the locomotion of a robot through treacherous terrain. 
This means the development of the scientific vision system 
for the robot is effectively decoupled from the development 
of the locomotion system for the robot, which has certain 
advantages. 

After the maturation of the computer- vision algorithms, 
we hope to transplant these algorithms from the cyborg com- 
puter to the on-board computer of a semi-autonomous robot 
that will be bound for the red planet. 

The digital hardware of the Cyborg Astrobiologist system 
consists of a 667 MHz wearable computer (from ViA Com- 
puter Systems in Minnesota) with a 'power-saving' Trans- 
meta 'Crusoe' CPU, a Head Mounted Display (fromTekgear 
in Virginia, via the Spanish supplier Decom in Valencia) 
with native pixel dimensions of 640 by 480 that works well 
in bright sunlight, thumb operated hand mouse, and a SONY 
'Handy cam' color video camera. The power- saving aspect 
of the Crusoe processor is important because it extends bat- 
tery life (Figure 3), meaning that the human does not need 
to carry very many spare batteries, meaning less fatigue for 
the human geologist. A single battery, which weighs about 
1kg, can last for 3 hours or so for this application. 

During the live demonstration of the Cyborg Astrobiolo- 
gist system we put a large poster in the front of the lecture 
hall of the mosaic image from the Mars Pathfinder site. The 
lecturer disconnected the VGA output of the wearable com- 
puter from the TekGear Head-Mounted Display (HMD), and 
then reconnected the VGA output of the ViA wearable com- 
puter to the LCD projection system of the auditorium. A 
SONY Handycam provided real-time imagery of the poster 




Fig. 4. From imagery from the Rio Tinto drilling campaign, we 
compute edges in 4 different directions (top row); hue, saturation 
and intensity (H, S, and I, 2nd row); image- segmentation based 
upon H,S and I (3rd row); and uncommon maps based upon the 
segmentation of H, S, and I (4th row). 




Fig. 5. Using the derived images in Figure 4, from the original 
color image on the right, we add the derived images together with 
certain coefficients, in order to determine the cortical interest map, 
shown on the left. The coefficients automatically adapt, depending 
on the images already processed. 

of the Mars Pathfinder site to the wearable computer via 
an IEEE1394/Firewire communication cable. The wearable 
computer processed the images, to compute a map of in- 
teresting areas (Figure 5), at a rate of about 1 image per 8 
seconds. This is rather slow, but considering that the system 
was doing a lot of computations, it certainly is not unreason- 
ably slow. The computations included: two-point intensity- 
correlations for image- segmentation (Figure 4, Haralick, 
Shanmugan & Dinstein, 1973; Haddon & Boyce, 1990), 
edge-computation using Sobel operators (Figure 4), and adap- 
tive fusion of 9-1 1 different derived images computed from 
the original color image (Figure 4) into a single 'cortical' 
interest map (Figure 5, Fislage, Rae & Ritter, 2000). We 
also used the image- segmentation map to compute an 'un- 
common' map, which gives highest weight to those regions 
of smallest area (Figure 4). 

We tried to point out during the demonstration how the 
system, when pointed at a poster from the Mars Pathfinder 
site, was able to find certain areas of the images interesting, 
for example the edges of the hill on the horizon, the shadow 
under one of the nearby rocks, or some of the rocks that had 
more saturated colors than the rest of the image. 

3 We believe in NEO 

For programming the Cyborg Astrobiologist Computer sys- 
tem, we use a Graphical Programming language, called NEO, 
from the University of Bielefeld (Germany), (Ritter et al 
2000). The 'Neural EditOr' (NEO) has such capabilities 
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Fig. 6. A NEO program, and the Matrix character Neo as an infant. 
The wires serve as inputs and outputs both to Neo and to the NEO 
icon sub-programs. 



that the programmer can write code rather rapidly and vi- 
sually, by removing most of the tedious aspects out of pro- 
gramming. It does this by replacing many keystrokes with 
several mouse clicks, and by graphically showing the pro- 
gram and its flow. A subroutine is replaced by a graphical 
icon with input and output pins of varying types. And dif- 
ferent icons can be connected by wires to pass data between 
two icons, from the output of one to the input of the next 
(see Figure 6). A NEO icon can be: 1) a container contain- 
ing graphical NEO subcircuits, 2) a short C or C++ subrou- 
tine written by the user and 'interpreted' by NEO, or 3) a 
more complex C or C++ subroutine that is precompiled for 
speed enhancement. 

One example of the power of NEO is that the whole pro- 
gram does not need to be run; a single icon can be executed 
or modified without running the rest of the program. There- 
fore, if the program is misbehaving and doing things that 
the Human-Geologist trainer of the NEO circuit finds to be 
inadequate (see Figure 7 for a symbolic setting of a Human 
Geologist training a NEO Cyborg Geologist circuit), then 
the NEO circuit can then be stopped after a certain number 
of executions of the whole circuit, and one or more of the 
individual NEO icons (sub-circuits) can then be executed 
without running the whole program. This is especially use- 
ful since the inputs to these NEO sub-circuits are frozen; 
hence, if a bug exists, the user can rerun the faulty sub- 
circuits many times, and the user can even make changes 
to and eventually correct the faulty NEO sub-circuit, with- 
out recompiling the whole program and even keeping the 
inputs frozen (recall the 'deja vu' black-cat events in the 
Matrix Movie, as in Figure 8). 

Eventually, and rather optimistically, the ability of the 
Human Programmer or the Human Geologist to find and 




Fig. 7. Cyborg geologist training program. For symbolic purposes, 
we refer to the man on the left (Morpheus) to be the Human Ge- 
ologist, and the man on the right (Neo) to be the computer- vision 
algorithm, which is currently being trained by the Human Geolo- 
gist. 




Fig. 8. Part of the Matrix program which is repeated a second 
later (a deja vu event), and which signified to the team that the 
programmer was making change to the program, to correct a per- 
ceived flaw. This ability to make debugging changes and view 
their results without restarting the whole program is one of the key 
capabilities of the NEO graphical-programming language. 



correct faults in the performance of the NEO Cyborg Geol- 
ogist circuit will diminish (see Figure 9). This improvement 
in the abilities of the NEO Geologist program may even 
someday allow the Cyborg Geologist to surpass the abili- 
ties of the original Human Geologist trainer (see Figures 10 
and 11). 

But more realistically, much work and preparation needs 
to be completed before the first mostly-autonomous systems 
can be trusted to go to Mars (Figure 12). 



4 Outlook 

The NEO programming for this Cyborg Geologist project 
was initiated with the SONY Handycam in April 2002. The 
wearable computer arrived in June 2003, and the head mount- 
ed display arrived in November 2003. We have a function- 
ing human and hardware and software Cyborg Geologist 
system now. We have even extended the system somewhat 
by connecting the serial port of the wearable computer to a 
Pan Tilt mount, so that the camera can be pointed in differ- 
ent directions, automatically by the computer. 

With the functioning Cyborg Geologist system, we plan 
extensive field tests in early 2004. The Cyborg Geologist 
system will test the basic capabilities of our image- segment- 
ation and interest-map algorithms on the grounds of our 
Spanish research institute, which is surrounded by Span- 



Fig. 9. Cyborg Geologist beginning to learn from the Human Ge- 
ologist. 




Fig. 10. Cyborg Geologist beginning to defeat the Human Geolo- 
gist 




Fig. 11. Cyborg Geologist beating Human Geologist. 

ish prairie with a small lake nearby. And in the same time 
frame, the Cyborg Geologist system will accompany a field 
geologist to sites of more geological interest (i.e. layered 
rock outcrops in the Castillian or Guadalajaran mountains) 
for more advanced testing of these computer- vision algo- 
rithms. 

5 Query after the talk 

Query: Do you think a system could be developed that 
could properly take into account the judgements of the ro- 
bots, astronauts, ground-controllers and administrators, when 
it comes to making decisions in the future exploration of 
Mars? (asked by a dark-haired, bespectacled gentleman in 
the right half of the audience; our apologies if his question 
was misinterpreted.) 

Response: Hmmmm. The answer to your question is an- 
other question: (answered seriously, but in terms of the sym- 
bolism and language of the two characters, the Oracle and 



Fig. 12. Cyborg Geologist intelligence trying to replace the intel- 
ligence of a Human Geologist before the Cyborg Geologist intel- 
ligence is completely ready, causing a fall when it tries to make its 
first jump from Earth to Mars. 

the Architect, in the Matrix film) "Would you really want to 
implement such a system?" pause 

In order to do that, you would need an 'Architect' . pause 
There is a technique ( answered more seriously, and with- 
out intentional symbolism from the lecturer) in Artificial In- 
telligence programming that could be used for this applica- 
tion, called "Modular Neural Networks" (cf. Haykin 1990), 
wherein the judgements of individual subprogram modules 
(i.e. the individual judgements of a robot, an astronaut, a 
ground-controller or a NASA administrator) are all com- 
bined into one master judgement. The Modular Neural Net- 
work learns from past mistakes, and its errors are passed 
down to correct the 'master weights' (of the connections 
from judgements of the individual modules to the master 
judgement), and also to correct the internal workings of the 
the individual modules themselves. 
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